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CHAPTER I 


INTRODUCTION 

During recent years, a number of investigators 
(e.g., Borgman, I960; Russo, 1966; Thompson, 1968) have 
devoted attention to methods of improving the utility and 
economic value of weather forecasts. As a corollary to 
such studies, procedures for making useful decisions in 
the face of uncertainty have become of increasing interest 
to the meteorologist, and many concepts of mathematical 
decision theory have been applied to this problem (Gleeson, 
I960; Glahn, 1964; Thompson, 1966). 

In general, the purpose of such studies has been 
to provide methods for optimizing the use of weather 
information, and to consider the economic or operational 
consequences of a certain decision tactic when applied to 
a specific operation. In this study, it is proposed to 
examine how differing decision models and operational 
characteristics affect the economic expenses (i.e., the 
costs of protection and losses suffered if no protective 
measures have been taken) associated with the use of 
predictive weather information. 



CHAPTER II 


METHOD OP APPROACH 

Weather Information and Decision Making 

Inherent in the nature of weather information is 
the characteristic of uncertainty. This arises from the 
lack of totally accurate knowledge of the initial state 
of the atmosphere, as well as from the inability to 
formulate and solve in complete detail the equations 
which describe how one state of the atmosphere develops 
from another. Even though better observing and forecasting 
methods will gradually alleviate these difficulties, it is 
likely that this factor of uncertainty will continue to be 
of importance throughout the forseeable future. Therefore, 
when the expenses resulting from the use of weather 
information are evaluated quantitatively, the nature and 
magnitude of meteorological uncertainty must be considered. 

The uncertainty in weather forecasts may be expressed 
conveniently through the language of probability. While 
some theoretical differences exist concerning the interpre- 
tation of this language, this study will take the practical 
point of view that the probability of occurrence of an 
event is equivalent to its relative frequency of occurrence. 
In making use of this concept, it is proposed to consider 
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first the simple dichotomous decision problem: Whether 

or not to protect against the occurrence of adverse 
weather. 

Dichotomous Decisions — Theory 

It has been shown by Thompson and Brier (1955) 
that optimum long-run decisions will be obtained for 
repetitive dichotomous operations, if protective measures 
are taken according to the following criterion: 

> Protect 

P = C/L Either course (1) 

< Do not protect 

is the probability of occurrence of adverse weather, 
is the cost involved on each occasion that protective 
measures are taken, and 

is the loss suffered on each occasion that adverse 
weather occurs and no protection is provided. 

This decision procedure is designed to minimize the long- 
run expense (i.e., cost of protection, plus loss suffered 
if no protection is provided and adverse weather occurs) 
incurred by the forecast user. Thus, the decision is 
specified once the value of the operational risk ratio, 

C/L, is ascertained. For a series of such decisions, the 
results may be represented as shown in Table I. 


where P 
C 

L 



a 

TABLE I. Generalized contingency table relating 
decisions for protective measures and observed 
frequencies (a, b, c, d) of the occurrence (W) 
and non-occurrence (No W) of adverse weather. 

Decision 
Do not 




protect 

Protect 

Total 

Observed 

No W 

a 

b 

a + b 

weather 

W 

c 

d 

c + d 


Total 

a + c 

b + d 

N « a+b+c+< 


Assuming that Table I is based on equation (1), the 
total expense for protecting against adverse weather will 
be due to the cost of protection, C, multiplied by the 
frequency with which such protective measures are taken, 
plus the loss, L, multiplied by the frequency with which 
adverse weather occurs and protective measures are not 
provided. Thus, E the total weather expense for optimum 
long-run decisions, is given by 


E 


opt 


= C (b + d) + L c 


( 2 ) 
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As a matter of convenience, E as well as all 

* opt’ 

other expenses, are represented in this study in "non- 
dimensional" form (i.e., as the expense per unit forecast 
per unit of loss). Therefore, 

E * = (C/L) (b + d)/N + c/N . (2a) 

opt 

However, if operationally "perfect" forecasts 
were attainable, then the only expense would be due to 
the cost of protection, multiplied by the number of 
occurrences of adverse weather. The non-dimensional form 
of such expenses, Ep er ’» is thus: 

Eper ’ * (C/D < c + d )/ N • (3) 

For the most part, probability information is not 
made available to the decision maker (forecast user), but 
instead categorical predictions of the occurrence or 
non-occurrence of adverse weather are issued. Since these 
forecasts must be used by the entire community, they are 
based on a quasi "average" operational risk ratio. * 


*In this study, it is assumed that the categorically 
predicted weather event is that which is most likely to 
occur (i.e., the modal value of the probability distribution). 
For a dichotomous problem, this is the weather event which 
is predicted with a probability exceeding 0.5; thus 
inferentially , the "average" operational risk ratio (C/L) 
is assumed by the forecaster to have this value. 
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Thus the expense associated with the use of such predictions, 
E cat’> ls: 

E ca t ’ * (C/L) (b a + d a )/N + C a /N » 

where the subscript "a” denotes the fact that the 
categorical forecast is made at an "average" decision 
level. 

A different expense is incurred by the operator 
with limited capital, who wishes to minimize the likelihood 
of experiencing an undesirable sequence of weather-caused 
losses. An appropriate strategy for this situation would, 
in general, involve taking protective measures more often 
than with the optimum long-run strategy. As noted by 
Gleeson (i 960 ) and Thompson (1966), this alternate strategy 
may be carried out by the use of a second-order estimate 
of the meteorological uncertainty, in which a lower limit 
of confidence is applied to each decision. Such a strategy 
produces the effect of minimizing the likelihood of 
incurring maximum losses for short-run periods. In decision 
theory terminology, this is a form of "mini-max" strategy. 

The expense associated with this strategy, E rn ^ n ', is: 

E min' - < C/L > < b m + d m> + V > (5) 

where the subscript "m" denotes the use of the "mini-max" 


tactic . 
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In similar manner, a procedure in which protective 
measures are taken less often than is the case for the 
optimum long-run strategy, may be specified. Here upper 
confidence limits are determined, and protective measures 
are implemented only when the forecast probability is 
greater than, or equal to, the upper limit at the 
appropriate operational risk ratio. Such a procedure 
maximizes the likelihood of incurring minimum economic 
expenses, and is called in this study a "maxi-min" strategy. 
The expense associated with this strategy is computed as 
in equation (5), but with the respective frequencies 
represented by the occurrence of weather events prescribed 
by decisions made at the upper confidence limits. 

Before examining a more complex decision problem, 
it should be noted that the equations (2)-(5), which 
express the expenses associated with the various dichotomous 
decision strategies, involve only the use of the forecast 
verification frequencies and the operational risk ratios. 
Therefore, these equations are quite general and can be 
applied to weather dependent operations in all aspects of 
the economy, where simple dichotomous decisions are 


involved. 
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Multiple Category Decisions — Theory 

Of greater complexity than the dichotomous situation 
are those problems which involve decisions concerning 
whether or not to take greater protective measures as the 
severity of the predicted adverse weather increases. Such 
problems require that the user provide a "utility matrix", 
in which the economic expenses (i.e., costs and losses 
representing a decrease in profits) associated with the 
decisions and resulting weather events, are specified. 

Using a notation similar to that of Gleeson (I960), the 
general structure of such a matrix is shown in Table II, 
where the expenses, > are expressed, as before, in 
"non-dimensional" units. 


TABLE II. Economic expense, a , for various 
decisions, D^, and observed weather events, 

V 


Weather Events 


Decisions D, 


m 


11 

► • • • a *••••< 
j 

• • • • a 

lk 


^ ^ ^ r 


il 


.... a lk 


o 

» • • • a 

l ml 

mj 

mk 
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Various decision strategies may be evaluated by 
applying the methods of mathematical decision theory to 
the values of a^j in Table II. For each of these 
strategies, the theoretical expense corresponding to each 
decision, D^, is computed. Then, if desired, the decision 
which corresponds to the lowest expense may be selected. 
(These theoretical expenses are equivalent to the actual 
economic expenses only if the assumed probability 
distributions are realized.) 

As an illustration of this concept, a Bayes strategy 
(Chernoff and Moses, 1959), which, in decision theory 
terminology, is equivalent to an optimum long-run strategy, 
may be applied to the expense values in Table II. Here, 
the Bayes expense, f , associated with each decision is 
defined by the following equation: 


V 


k 
I a 
J=1 


ij p j 


( 6 ) 


where p is the forecast probability of occurrence of 
J 

observed weather event, X . A decision which is based on 
a minimum value of E^*, assures the forecast user that if 
over a sufficient time span the predicted probability 
distribution is realized, then his actual economic expense 
will be a minimum. 
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In similar fashion, the decisions corresponding to 
perfect forecasts can be determined. However, the 
probability, pj, takes on a value of unity for observed 
events and zero for all others. 

In order to provide a rational basis for making 
a "categorical prediction" (see page 5)» the weather event 
which is most likely to occur (i.e., the modal value of 
the probability distribution) is again considered as being 
the categorically predicted event. 

A decision resulting from the application of a 

Bayesian "mini-max" tactic, can be determined by a 

procedure proposed by Gleeson (1960).^ Upper and lower 

confidence limits, Pj" and P j ’ » are evaluated for the 

forecast probabilities, Pj , associated with each weather 

event, X , subject to the restriction, 

J 

1 = (p " - Pj') = 0 . (7) 


p 

Since Gleeson was concerned with the economic 
expectation, instead of the economic expense, the notation 
used in this study for the multi-category "mini-max" and 
"maxi-min" strategies is the reverse of that which appears 
in Gleeson* s article. 
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In order to implement the "mini-max" tactic, the maximum 
expense associated with each decision, e^,'^ 1113 *^, may be 
determined by using the following equation: 

, (max) „ £ Pjj (max) ^ (8) 

where p ^^ max ^ is a probability which is derived for each 
weather event. This derived probability is a maximum or 
minimum when a^ is maximum or minimum, respectively, such 
that, 


P J 


tt 


n (max) 
P ij 



(9) 


The "mini-max" decision is the one for which e^'^" 13 *) is 
minimum . 

In analogous fashion, a "maxi-min" strategy 
(Thompson, 1966) may be considered. Selecting the 
appropriate decision for this strategy involves the 
evaluation of the minimum economic expense, . 

Here, 


p i (min) = £ (min) 

I j*l iJ iJ 


( 10 ) 
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where p ^( min ) is a maximum or minimum when a^ is a 
minimum or maximum, respectively. The decision for which 
g^f(min) i owes t i s selected as the "maxi-min" decision. 

The final expense which will be considered in 
connection with the multi-category decision problem, is 
produced by either a quasi "mini-max" (play it safe) 
tactic or a quasi "maxi-min" (gambling) tactic in those 
situations when the user is provided only with categorical 
predictions. Here, several alternatives are available for 
selecting appropriate decisions. A "play it safe" procedure, 
which would seem to be adequate for the purposes of this 
analysis, is one involving the implementation of protective 
measures as if adverse weather of one category more severe 
than is predicted by the categorical prediction will occur. 

In similar fashion, a suitable "gambling" tactic would 
involve the taking of protective measures as if adverse 
weather of one category less severe than is predicted by 
the categorical prediction will occur. Thus, once the 
categorical forecast is obtained, the appropriate decisions 
may be specified. ^ 

^The "play it safe" and "gambling" strategies were 
not considered for the dichotomous decision problem, since 
comparable procedures would involve the implementation of 
trivial courses of action (i.e., either protecting against 
adverse weather all of the time or not protecting at all). 
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Of interest is the mean economic expense, E * , which 

5 

is associated with each of the multi-category strategies 
for which the decision procedures have been outlined. In 
order to evaluate this expense, it is necessary to compute, 

V ■ ^ . (11 > 

where "N" represents the total number of forecasts, 

"a^ ” is the "non-dimensional” expense (Table II, page 8), 
and the subscripts are defined as follows: "s" denotes 

the particular decision strategy, "d" refers to the decision 
which was implemented, and "w" indicates the weather event 
which actually occurred. 

In summary, it has been shown that the methods for 
determining the economic expenses corresponding to multiple 
option and simple dichotomous decision problems are 
different. In particular, while the dichotomous situation 
involves the use of forecast verification data and the 
users* operational risk ratios, the multi-category 
strategies require that the expense associated with each 
individual forecast be determined in relation to specific 
operational requirements. 



CHAPTER III 


ECONOMIC EXPENSES— SPECIFIC EXAMPLES 
Dichotomous Decisions 

It Is of interest to evaluate the expenses 
associated with the various dichotomous decision strategies. 
In order to accomplish this task, U.S. National Weather 
Service forecast verification data were collected and 
analyzed. Data included: (1) short-period (3-, 5 -$ and 

7-hour) predictions of aviation terminal weather for 
8 major airports across the United States (Allen, 1969), 

(2) medium-range (12-, 2M-, and 36-hour) precipitation 
forecasts at 22 cities throughout the nation (Russo, 
et al., 1967), and (3) extended-period (5-» 30- , and 
90-day) temperature and precipitation predictions for the 
entire United States (Namias, 1964 ) . 

Based on the dichotomous meteorologic-economic 
model which has been presented (pages 3-7), the raw data 
were processed with the aid of an electronic computer. 
Examples of the results which were obtained are presented 
in Figs. 1 and 2. In each case, the economic expense is 

il 

plotted against the operational risk ratio. The 

^The expenses corresponding to the implementation of 
the dichotomous "maxi-min" strategy are not presented 
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FIG. 1. Expenses associated with differing dichotomous 
decision strategies for 24-hour precipitation forecasts 
(Boston, Massachusetts, April-September , 1967, inclusive). 
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PIG. 2. Expenses associated with differing dichotomous 
decision strategies for 5-hour ceiling forecasts (San 
Francisco, California, September, 1965-March, 1966, 
inclusive) . 
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confidence limits, necessary for the evaluation of the 
"mini-max" strategy expense (page 6), were obtained by 
a method outlined in Appendix A. 

The expenses which result from applying the various 
dichotomous decision strategies to a series of precipitation 
forecasts at Boston, Massachusetts, are depicted in Fig. 1. 
The expense associated with the use of perfect forecasts 
is included for reference. The optimum long-run strategy . 
appears to produce the least expense for the majority of 
the operational risk ratios (C/L). It is also apparent 
that the expenses produced by categorical and perfect 
forecasts, which both do not require the use of a decision 
strategy, are linear functions of the ratio C/L. This is 
because the frequencies (a, b, c, d. Table I, page M) are 
constant for all values of C/L [see equations (3) and (*0] 
when categorical or perfect forecasts are used in this 
manner. However, the equations [(2a) and (5)] which 
correspond to the other strategies (optimum long-run and 
"mini-max") make use of differing frequencies with each 


since: (1) this extremely hazardous strategy is seldom 

used in practice, (2) excessive amounts of computer time 
are involved in the evaluation of the expenses, and 
(3) several examples of the expenses associated with this 
strategy are considered in regard to the more interesting 
multiple option decision problem. 
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value of C/L, and the associated expenses are non-linear. 

In addition, the observation that the optimum and "mini-max" 
strategies produce the same expense as the perfect forecasts 
at C/L * 0 is fortuitous (see Pig. 2). In practice, of 
course, no forecasts would be needed at the C/L limits, 
since at C/L « 0 there is no cost of protection and at 
C/L = 1 the cost of protection is the same as the loss. 

Pig. 2 depicts an analysis of short-range ceiling 
forecasts at San Francisco, California. In general, this 
example exhibits characteristics which are similar to those 
observed in Pig. 1. However, the magnitude of all expenses 
are much lower in Fig. 2, since these forecasts cover a 
shorter time span (5 hours) than those used for Pig. 1 
(24 hours) . 

Multiple Category Decisions 

As a means of illustrating how the multi-category 
decision strategies are used in a practical manner, two 
utility matrices. Tables III and V, were developed for 
weather dependent operations. In each matrix the numerical 
value of the economic expense corresponds to a given forecast 
and observed weather event. Thus, once the forecast is 
specified, a decision can be determined by some appropriate 
method. The economic expenses in each matrix are based 
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on an arbitrary scale ranging from zero to unity. The 
categories of weather events had previously been selected 
by the U.S. National Weather Service, which also provided 
the forecast verification data. 


TABLE III. Relative economic expense due to 
the prediction and/or occurrence of indicated 
categories of ceiling and visibility. 


Forecast 

Category 


Observed 


Category 


1 

.70 

.60 

.65 

.70 

.75 

2 

.90 

.40 

• 30 

.25 

.10 

3 

.95 

.40 

.30 

.25 

.05 

4 

• 95 

.45 

.35 

.20 

.05 

5 

1.00 

.50 

.40 

.30 

0 


Explanation of categories: 


Ceiling 

Category ( feet) 


Visibility 

(miles) 


1 

2 

3 

H 

5 


IlOO 
200-400 
500-900 
1000-2900 
= 3000 


= 3/8 

1 / 2-1 3/8 
1 1/2-2 1/2 
3-4 
>5 
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A utility matrix for a generalized aviation landing 

5 

and take-off operation is presented in Table III. In 
this matrix the values of the economic expenses represent 
"mean" estimates for a number of varied operations. 

Therefore, some adjustment in these values may be expected 
for differing individual circumstances. A list of the 
operational basis for the various combinations of forecast 
and observed weather event for this matrix is contained 
in Appendix B. 

By using short-period (3-, 5 -» and 7-hour) forecasts 
of ceiling and visibility in conjunction with the utility 
matrix in Table 3, and implementing the methodology which 
has been described (pages 8-13), an analysis of the 
multiple category decision strategies was conducted. An 
example of the results is presented in Table IV. 

An examination of Table IV reveals the same general 
expense pattern as was observed for the dichotomous decision 
strategies. The expenses associated with the use of perfect 
forecasts are lowest, while the other strategies produce 
expenses of similar magnitudes; except for the relatively 

^The matrix was developed by Professor Gerald Shreve 
of the Aeronautics Department, California State University, 
San Jose, in consultation with a number of private, business, 
and commercial aircraft operators. 
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large values associated with the "play it safe" strategy. 

Of additional interest, is the fact that the expenses 
produced by the ceiling forecasts are of greater magnitude 
than those associated with visibility predictions. This 
arose because low ceilings occurred more often than adverse 
visibilities during this time period. 

In analogous fashion. Table V depicts a utility 
matrix which was used in connection with 2M-hour quanti- 
tative precipitation forecasts.^ An examination of Table V 
reveals that for the construction operation to which this 
matrix applies, the losses accumulate rapidly as the 
rainfall rate increases. 

The expenses which result when a precipitation 
forecasting system (Thompson, 1950) is applied to this 
matrix are listed in Table VI. Here the economic expenses 
are, in general, of small magnitude (i.e., 1 to 5 per 
cent of the mean protectable losses). This is to be 
expected, since the majority of the expense values in the 
utility matrix (Table V) are relatively small. Thus, for 

^The matrix was developed by Dr. R. Robert Rapp, 
Certified Consulting Meteorologist, in regard to a 
construction operation in the Los Angeles area. 
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the construction operation under consideration, this 
collection of precipitation forecasts appears to be quite 
useful, since the undesirable situation where little or 
no precipitation is forecast and heavy precipitation obtains, 
happens infrequently. 


TABLE V. Relative economic expense due to the 
prediction and/or occurrence of indicated 
categories of precipitation. 


Forecast 

Category 

1 

Observed 

2 

Category 
3 4 

5 

1 

0 

.024 

.119 

.476 

1 

2 

.008 

.008 

.103 

.460 

.974 

3 

.040 

.040 

.040 

.396 

.920 

4 

.159 

.159 

.159 

.159 

.682 

5 

• 333 

.333 

.333 

.333 

.333 


Explanation of categories: 


Category 

1 

2 

3 

4 

5 


Precipitation (Inches) 

<.01 
.01-. 15 
. 16 -. 1*9 
.50-1.50 
> 1.50 
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CHAPTER IV 


DISCUSSION OP RESULTS 
Dichotomous Decisions 

Because of the generality of the dichotomous 
decision strategies, an analysis of the economic expenses 
for the entire economy may be carried out. If it is 
assumed that all operations are equally likely and equally 
important, simple average values of the economic expense 
may be computed, thus providing a summary of the results. 

Such mean values are shown in this chapter. 

Of interest, is the manner in which the expenses 
associated with the various strategies fluctuate in response 
to changes in weather elements and length of forecast period. 
Since the prediction of low visibility is of increasing 
importance to modern aircraft operations, the results of 
the expense computations for 5-hour forecasts of visibility, 
where the "adverse” weather is visibility 2 1/2 miles or 
lower, are presented in Pig. 3» Complete data for 5-hour 
ceiling and visibility forecasts are listed in Table XII 
(Appendix C, page 56 ). The curves in Pig. 3 show that 
average expenses associated with the differing strategies 
at the 8 major air terminals under consideration. 
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FIG. 3. Expenses associated with differing dichotomous 
decision strategies for 5-hour visibility forecasts 
(September, 1965-March, 1966, inclusive) 
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range from 3 to 11 per cent of the protectable losses. 

Thus, in percentage terms, these expenses are relatively 
small, which is presumably due to the short-range nature 
of the forecasts. 

Regardless of the decision strategy, it may be 
noted that the expenses in Pig. 3 at Los Angeles International 
Airport are the largest of all airports shown, while those 
at San Francisco International Airport are the smallest. 

In part at least, this is due to the fact that Los Angeles 
had over twice as many occurrences of low visibility as 
did San Francisco. Possible causes for this difference in 
low visibility frequency are: (1) airport location 

(i.e., proximity to the ocean), (2) trajectory of the 
prevailing winds, (3) local sources of pollution, and/or 
(4) the advection of inland nocturnal radiation fogs. 

Medium-range predictions were also examined and 
Figs. 4-9 depict the manner in which the economic expenses 
associated with 24-hour precipitation forecasts are 
influenced by seasonal weather variations. 

Fig. 4 illustrates the "winter" season pattern of 
precipitation frequency for the United States. It is 
apparent that the greatest frequencies occur in the 
Pacific Northwest, while a secondary maximum is present 
near the Great Lakes. In contrast, an area of minimum 
precipitation frequency is located in the Southwest. 




PIG. H. Number of days with measurable precipitation (> 0.01 inch) for "winter 
season (October, 1967-March, 1968, inclusive). 
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FIG. 5. Mean probability scores for "winter” season (October, 1967-March, 1968 
inclusive) for 24-hour precipitation forecasts. 








30 



FIG. 6. Dichotomous decisions — optimum long-run strategy mean expense values 
for "winter" season (October, 1967-March, 1968, inclusive) using 2M-hour precipitation 
forecasts. Units are per cent of protectable loss. 





PIG. 7 . Mean number of days with measurable precipitation (* 0.01 inch) for 
summer" season ( April-Sept ember , inclusive, 1967 and 1968). 
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PIG. 8. Mean probability scores for "summer" season (April-Sept ember , inclusive 
1967 and 1968) for 2 1 »-hour precipitation forecasts. 
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PIG. 9. Dichotomous decisions — optimum long-run strategy mean expense values for 
ler" season ( April-September , inclusive, 1967 and 1968) using 24-hour precipitation 
asts. Units are per cent of protectable loss. 
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In similar fashion, mean probability scores 

(Brier, 1950), which gives an indication of the general 

7 

"accuracy" of the forecasts, are shown in Fig. 5. A 
comparison of Fig. 5 with Fig. 4, indicates that the 
forecasts are less accurate in areas of high precipitation 
frequency . 

Fig. 6 reveals the pattern of the mean expenses 

which are associated with the use of the optimum long-run 

* 

decision strategy. Here, the isopleths of expense produce 
patterns which are similar to those presented in Figs. H 
and 5. This is reasonable, since the expense is dependent 
upon both the occurrence of adverse weather and the 
accuracy of the forecasts. In addition, the mean expenses 
for the "winter" season corresponding to the implementation 
of the other dichotomous decision strategies (i.e., perfect 
forecasts, "mini-max", categorical forecasts) exhibit the 
same general characteristics as the optimum strategy 
(Fig. 6). 


^The accuracy in this case is a function of both 
the "reliability" of the probability estimates (i.e., the 
relationship between each probability statement and the 
subsequent relative frequency of occurrence), and their 
"resolution" (i.e., the closeness to zero or unity of 
each probability statement). 



The "summer" season pattern of precipitation 
frequency. Fig. 7, demonstrates the manner in which 
seasonal weather variations influence the economic expense 
values. The primary maximum is located in the southeastern 
United States, where random showers and thunderstorms are 
a frequent phenomenon. Consequently, the mean probability 
scores, shown in Fig. 8, reach the largest values (i.e., 
the forecasts are least "accurate") in this region. 

An examination of the "summer" season mean expenses 
produced by the optimum long-run strategy. Fig. 9, indicates 
the similarity between the pattern of expense and the 
precipitation frequency. As is the case with the data 
for the "winter" season, further analysis shows that 
the other dichotomous decision strategies produce expense 
patterns which are similar to those associated with the 
optimum strategy. 

Confirmation of the supposition that the average 
expenses are closely related to both the precipitation 
frequency and accuracy of the forecasts, was obtained by 
computing the respective linear correlation coefficients. 

The results of this analysis are listed in Table VII. All 
of the correlation coefficients are quite large and all 
are significant at the 1 per cent level, according to an 
analysis of variance which was conducted. 
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The final application of the various dichotomous 
decision strategies involved the use of extended-range 
categorical predictions of temperature and precipitation 
for the United States as a whole. Since a priori 
(conditional) probability statements were not available, 
the probability information for these forecasts had to be 
obtained from a posteriori (verification) data. As a 
consequence, these latter probabilities may be less "sharp" 
in the statistical sense, than the individual conditional 
probability statements associated with the other forecasts 
which have been considered. 

For these extended predictions. Fig. 10 illustrates 
the variation in the expense with increase in the length 
of the forecast period. It is evident that the expenses 
associated with the various strategies become larger as 
the forecast period increases. In particular, the expenses 
associated with the seasonal (90-day) predictions range 
from approximately 15 to 35 per cent of the mean protectable 
losses. In addition. Fig. 10 shows that the expenses for 
the differing strategies demonstrate large deviations in 
magnitude for the extended-range predictions. 

In connection with the categorical statement of 
seasonal predictions, Namias (196*4) has shown that 
forecasts of this nature possess marginal positive skill 




PIG. 10. Expenses associated with various 
dichotomous decision strategies In relation to the 
length of forecast period. 
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with respect to "climatological" forecasts (i.e., predicting 
the weather in the same ratio as would be obtained by chance, 
using the climatological expectancy). In order to 
investigate the contrast in economic utility between the 
a posteriori seasonal forecasts and climatological forecasts, 
an analysis of the expenses was conducted and the results 
are shown in Table VIII. Furthermore, the respective skill 
scores and the results of the statistical "chi-square" test 
are listed in Table IX. 

TABLE VIII. Dichotomous decision expense values 
(per cent of protectable losses) associated with 
a posteriori seasonal predictions of temperature 
and precipitation for the nation as a whole' (1959- 
1968), compared with the expenses produced by 
using climatological forecasts. 

Strategies 

Perfect Optimum Categorical 

Forecasts Long-run Mini-max Forecasts 

A Posteriori 

Seasonal 18.61 27.77 31 . 7 ^ 35.17 

Forecasts 


Climatological 

Forecasts 


18.03 


27.22 


30.50 


33.33 



TABLE IX. Skill scores and values of "chi-square" 


for categorical seasonal predictions of temperature 
and precipitation (1959-1968) as compared with 
climatological forecasts. 


Skill 

Score Chi- 
(?) square 


Categorical 

Seasonal 

Temperature 

Forecasts Vs. 

Climatological 

Forecasts 


9 15 .7^ 


Chi-square required for 
significance: 

1? 5? 10? 


13.28 


Categorical 

Seasonal 

jUMte i l 2-5 1-02 13.28 9.*9 7-78 

Climatological 

Forecasts 


Table VIII shows that the expenses produced by 
using climatological expectancy are slightly less than those 
associated with the seasonal predictions. Thus, it would 
appear that the particular set of a posteriori forecasts 
which were used in this analysis are actually of little 
economic value, compared to climatological predictions, 
when applied to the average of all operations for the 
entire economy. 
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The skill scores in Table IX, which were evaluated 
for the categorical predictions in comparison with 
climatological predictions, are in general agreement with 
the results of Namias (1964), i.e., 9 per cent for the 
temperature forecasts and 2.5 per cent for the precipitation 
forecasts. The values of "chi-square" show that while the 
categorical predictions of temperature are, at 1 per cent 
significance level, different than climatological forecasts, 
the predictions of precipitation are not. This result is 
also in agreement with the findings of Namias. 

Multiple Category Decisions 

While dichotomous decision problems clearly exist, 
it is apparent that certain operations require the 
consideration of multiple options. Therefore, in order 
to determine if the expenses for this type of problem 
differ markedly from those examined in the previous 
section, an analysis of the application of short-period 
ceiling and visibility forecasts to the generalized 
aviation matrix (page 19) was undertaken. The results 
of this study are presented in Figs. 11 and 12. 

The economic expenses corresponding to the various 
multi-category decision strategies for a series of 5-hour 
visibility forecasts are depicted in Fig. 11. Here, the 
expenses are comparable in magnitude (i.e., 3 to 13 per 
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PIG. 12. Expenses associated with differing rr >.j 1 1 5 — 
category decision strategies for 3-, 5-, and 7-hour ceiling 
and visibility forecasts (September, 1965-March, 1966, 
inclusive). 



cent of the protectable losses) to those which were 
presented in connection with the congruous dichotomous 
decision model (Fig. 3, page 26). A complete listing of 
the expenses produced by the multi-category decision 
strategies for the 5-hour predictions of ceiling and 
visibility is given in Table XIII (Appendix C, page 57). 

The heavy line on Fig. 11 shows that the magnitudes 
of the expenses produced by the various strategies differ 
very little. An exception is the "play it safe" strategy, 
for which the expense values are quite large. This 
suggests that the procedure used to implement this strategy 
(see page 12), while most likely a practical tactic when 
probability information is not available, may tend to 
produce considerably larger economic expenses than would 
be the case if such information were provided. Conversely, 
the "gambling" strategy, although introducing the possibility 
of substantial short-run losses, produces long-run expenses 
which are similar to those associated with the optimum 
strategy. However, it should be noted that these 
observations concerning the "play it safe" and "gambling" 
strategies are highly dependent on the structure of the 
particular utility matrix which was used to evaluate the 


expenses . 
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As a means of summarizing the expenses associated 
with the multiple option decision problem under consideration, 
Fig. 12 depicts the manner in which the expenses vary as the 
length of the forecast period changes. Here, the expenses 
produced by the differing strategies remain relatively 
constant as the forecast period ranges from 3 to 7 hours. 

Once again the expense corresponding to the use of perfect 
forecasts is lowest, while the other strategies, with the 
exception of the "play it safe", produce expenses of 
similar magnitude (approximately 6 per cent of the 
protectable losses). 

Total Dollar Losses 

Based on a meteorologic-economic model and 
implementing the concepts of mathematical decision theory, 
economic expenses have been computed for variations in 
users' decision options and strategies. In general, these 
variations appear to produce only small differences in the 
results for prediction periods of less than two days, but 
the differences increase to operationally significant 
values for monthly and seasonal prognoses. 

However, even small variations in economic expenses 
can be of importance when applied to the total protectable 
losses which are sustained by large segments of the 
economy. For example, the approximate magnitude of the 



annual total protectable losses for major activities in 
the United States are listed in Table X (Thompson, 1972). 
As shown in Table X, the total protectable weather-caused 
losses are substantial (i.e., over 5 billion dollars). 
Therefore, the expenses produced even by short-period 
predictions (i.e., 5 to 10 per cent of the total 
protectable losses), when expressed in actual dollar 
amounts, represent sizeable quantities of capital. For 
this reason, small differences between the economic 
expenses associated with various decision strategies can 
be of significance when applied to the economy of the 


United States. 
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TABLE X. Summary of annual total protectable 
weather losses for major activities in the 
United States 


Losses (millions of dollars) 
due to adverse weather which 
could be protected against if 
adequate warnings for the 
appropriate period in advance 
Activity could be provided. 


Agriculture 

3,554.2 

Aviation (commercial) 

56.9 

Construction 

328.6 

Communications 

6.4 

Electric Power 

13.9 

Energy (e.g., fossil) 
Fuels 

1.0 

Manufacturing (provisional 
estimate) 

238.0 

Transportation (rail, 
highway, and water) 

45.8 

Other (general public, 
government, etc.) 

1,057.8 


Totals 5,302.6 



CHAPTER V 


CONCLUSIONS 

The economic expenses which are produced when 
contrasting decision models and operational requirements 
are applied to various types of predictive weather 
information have been shown to exhibit several interesting 
characteristics . 

(1) The mean values for the dichotomous decision 
strategy expenses (per cent of protectable losses for 
the average of all operations) increase as the length of 
the forecast period increases (e.g., less than 10 per cent 
for predictions of up to 7 hours; 10 to 20 per cent for 
predictions of up to 36 hours; 15 to 35 per cent for 
predictions of up to 90 days. 

(2) While the differences between the mean expenses 
associated with various dichotomous decision strategies 

are of small magnitude (less than 5 per cent) for predictions 
of less than two days, the variations for monthly and 
seasonal forecasts are much larger (5 to 15 per cent). 

(3) The geographic variations in the mean expenses 
for dichotomous decision problems are closely related to 
the frequency of adverse weather and the accuracy of the 
predictions. 
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( <0 Dichotomous and multiple option decision 
problems produce mean expenses of similar magnitude (less 
than 10 per cent) for short period predictions. 

(5) Due to the large monetary value (over 5 billion 
dollars) of the total protectable weather-connected losses 
throughout the United States, even small variations in the 
economic expense values are of importance. 
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APPENDIX A 

CONFIDENCE LIMITS— METHOD OF DETERMINATION 

Lower and upper confidence limits are necessary for 
the implementation of the "mini-max” and "maxi-min" strategies 
for both dichotomous and multiple category decision problems. 
These limits were obtained by first computing the relative 
frequency of occurrence of adverse weather (e.g., precipi- 
tation, low airport ceiling and visibility) for a sample 
series of occurrences as a function of each class of 
predicted probability. Then the relative frequencies which 
included 90 per cent of the sample data were subjectively 
evaluated for each such class. These relative frequencies 
represent the 90 per cent lower and upper limits on the 
"confidence" associated with the probability estimates 
issued by the forecaster and/or forecasting system. 

A more sophisticated procedure was used to check 
the accuracy of the method outlined above. Assuming that 
the errors in the predicted probabilities are normally 
distributed, a normal curve was fitted to the sample data. 

The lower and upper confidence limits were then determined 
by standard statistical procedures (e.g., Panofsky and 
Brier, 1968). Analysis of the results revealed little 
difference between the confidence limits obtained by the 
two methods. 
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APPENDIX B 

EXPLANATION OF AVIATION UTILITY MATRIX 

TABLE XI . Summary of basis for relative values 
of the economic expense given in Table III. 



Observed 

Category 


Economic 

Expense 


Comment 


1 1 .70 Probable trip cancellation; 

if attempted, would require 
diversion. 


1 2 .60 Probable trip cancellation; 

if attempted, subject to ILS 
approach delays. 

1 3 .65 If trip cancelled, loss of 

revenue and intangible 
dissatisfaction since flight 
could be completed; if 
attempted, subject to some 
delays . 

1 4 .70 Same as previous comment, 

except if attempted, little 
or no delay. 


1 5 .75 Same as previous comment, 

except if attempted, no delay. 

2 1 .90 Diversion or holding probably 

anticipated; alternate planned 
and holding fuel carried, but 
serious delay encountered. 

2 2 .40 Same as previous comment, 

except no diversion necessary; 
delay due to ILS approaches. 



TABLE XI. (continued) 


Forecast 

Category 

Observed 

Category 

Economic 

Expense 

Comment 

2 

3 

.30 

Some traffic delay, but 
weather better than forecast; 
holding fuel carried. 

2 

4 

.25 

Same as previous comment, 
except less traffic delay. 

2 

5 

.10 

Traffic flow optimum, but 
holding fuel carried. 

3 

1 

.95 

Alternate probably not named; 
insufficient fuel for holding; 
must divert or land short of 
destination, but pilot might 
expect need for holding. 

3 

2 

.40 

Traffic flow reduced with ILS 
approaches; pilot may antici- 
pate need for holding. 

3 

3 

.30 

Weather observed as predicted, 
but considerable delay likely. 

3 

4 

.25 

Slight traffic delay, but 
holding fuel carried due to 
adverse weather forecast. 

3 

5 

.05 

Traffic movement optimum, but 
holding fuel carried. 

4 

1 

.95 

Alternate probably not named; 
insufficient fuel for holding; 
must divert or land short of 
destination, but pilot might 
expect need for holding. 

4 

2 

.45 

Traffic flow delay with 
unexpected ILS approaches; 
holding fuel necessary, but 
probably no diversion. 
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TABLE XI. (continued) 


Forecast Observed Economic 
Category Category Expense 


4 3 .35 

4 4 .20 

4 5 .05 

5 1 1.00 

5 2 .50 

5 3 .40 

5 4 .30 

5 5 0 


Comment 


Same as previous comment, but 
probably less traffic delay. 

Weather observed as predicted, 
but some traffic delay. 

Traffic movement optimum, but 
extra fuel carried due to 
slightly adverse forecast. 

Alternate probably not named; 
insufficient fuel for holding; 
must divert or land short of 
destination. 

Same as previous comment, 
except may be able to make 
destination after ILS delays. 

Same as previous comment, but 
may land with only general 
approach delays. 

Same as previous comment, but 
traffic moving well with visual 
approach used. 

Traffic movement optimum. 
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APPENDIX C 

ECONOMIC EXPENSES FOR 5-HOUR CEILING AND VISIBILITY FORECASTS 

TABLE XII. Dichotomous decisions — expenses 
associated with differing strategies for 
5-hour ceiling and visibility forecasts 
(September, 1965-March, 1966, inclusive). 

Units per cent of protectable loss. 

(C = Ceiling Forecasts, V * Visibility 
Forecasts ) 


Strate* 


Perfect Optimum Categorical 



Forecasts 

Long 

:-run 

Mini 

-max 

Forecasts 

Stations 

C 

V 

C 

V 

C 

V 

C 

V 

ALB 

2.9 

5.2 

4.6 

8.1 

5.3 

8.5 

5.5 

9.3 

JFK 

5.1 

4.2 

6.4 

6.2 

6.6 

6.6 

6.9 

7.4 

BAL 

4 . 2 

5.6 

5.4 

8.1 

5.5 

8.3 

6.4 

9.4 

DCA 

3.3 

3.1 

4.5 

4.6 

4.8 

5.0 

5.3 

5.2 

ORD 

2.1 

3.5 

3.7 

5.7 

4.3 

6.3 

4.8 

7.1 

SEA 

4.8 

5.5 

8.0 

8.2 

9.0 

8.9 

9.5 

9.3 

SFO 

2.5 

2.8 

4.2 

4.3 

4.7 

4.6 

5.0 

4.9 

LAX 

4.0 

6.3 

5.6 

9.0 

6.3 

9.7 

6.4 

10.6 


Stations used: 


ALB = Albany, New York 
JFK « New York City, New York 
BAL » Baltimore, Maryland 
DCA = Washington, D.C. 

ORD » Chicago, Illinois 
SEA = Seattle, Washington 


SFO = San Francisco, California 
LAX » Los Angeles, California 



57 


in o oo on vo co on on 








bd 

>i 








G 


vo vo o-^r in c^--=r o 







•H 


rH 







rH 









P 









£ 


vo on on -= r vo cvj oo 







cd 

ol 








O 


vo vo 43- -=r vo rHinvo 

T5 








rH 

0) 









P 

>> 








cd 

P 

CO 

•V 




>i 

av oco o iHc*-ao-=r 

•H 

•H 

p 

CO 





O 

rH 

•H 

p 



P 


ON O O OO CNrHC^-OJ 

O 

«H 

C 

CO 



•H 


H H rH rH 

CO 

P 

p 

cd 



CD 



00 

*H 


a 



>><H 



cd 

CO 


CD 



cd cd 

ol 

inrHaOOinCVJOOV 


•H 

• 

G 



rH CO 


GO 

> 

r-s, 

o 



Ph 


on o ao co crvcncNON 

CD 


CD 

PH 





rH rH 

CO 

X3 

> 







c 

c 

•H 

60 



rH 



CD 

cd 

CO 

C 



cd co 

>1 

cvj oo -=r rH -=r H h vo 

a 


G 

•H 



O P 


X 

bO 

rH 

rH 



iH CO 


vo inc^^r in co -=r on 

<D 

C 

O 

•H 



G cd 



i 

•H 

c 

CD 



o o 



i 

rH 

•H 

O 



be a> 


on onvo «h o ON-=r *h 

CO 

•H 





CD G 

ol 


C 

CD 

r> 

11 



P O 


vo in-=r ^rvooinvo 

o 

O 

VO 



CO 

Cd Ph 


rH 



vo 

O 


CD 

o 



CO 

G 

ON 

N / 




>1 

CVJ 0-^4 OJ -4 C“- rH OO 


3 

rH 



bC 

c 


o 

O 




CD 

•H 


vo in vo -=r in t^-=r on 

CD 

P 

*•> 

• 


P 

E 



T3 

1 

P 

CO 


cd 

l 




in 

O 

CO 


G 


ol 

CVONinOiHVOinrH 

>> 


G 

o 


p 

X 


G 

G 

cd 

rH 


CO 

cd 


vo in -4 ^rvoo invo 

O 

o 

2 




2 


rH 

bC 

<H 

l 

CD 






0) 


in 

rH 




>1 

o^r cvj oo -4 4- co 

p 

CO 

vo 

P 



X 


cd 

a> 

ON 

<d 

CO 


cd 


vovoc— -=r inoo -4 on 

o 

•H 

rH 

p 

P 


E 




bC 


o 

CO 


l 



<d 

<u 

•> 

CD 

cd 


-H 

Ol 

voovorHoncvjONm 

rH 

p 

G 

P 

o 


c 


a 

cd 

0) 

O 

CD 


•H 


vovo^r^rvorHmvo 

•H 

G 

p 

g 

G 


2 


rH 

P 

p 

E 

a 

o 





rH 

CO 

a> 


PH 



>1 

cvjoo cnj h in 



p 




G 



bC 

a 

o 

>> 


E 3 


vo mo-^r ino-^r on 


C 

0) 


P 


3 G 




•H 

CO 

p 

•H 


E 1 



• 

G 


G 

rH 


«H bC 

Ol 

on onvo o on on on o 

M 

0) 


CD 

•H 


P G 


M 


CO 

O 

P 


a o 


vo in -4 4- ino mvo 

M 


p 


«H 


O P 


rH 

X 

•H 

CO 

G 

CO 






'O 

cd 

CD 

•H 


C0 

>1 

OvmOVOUrHOOrHC^ 

w 


o 

CX 

> 


p p 


IP 

p 

<D 




O CO 


-4 -4 ionn4 moos 

CQ 

p 

G 

CD 

tl 


a> cd 



< 


O 

G 



V» O 




> 


cd 

> 


G 0) 


OV ONOO on C^-OO rH ON 







0) G 
CH O 

ol 

4 4 onon4 C"-4 -4" 







Ph 




w 

C 

o 

•H 

P 

cd 

p 

co 


OQ^ J<Q<OX 
^fo<ooc;wfc< 
CbPQQOWCOP 



